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ABSTRACT 


In this thesis we considered some tests which may 
be used to decide whether or not a given sample comes from 


exponential distribution with an unknown parameter. 


We first justified usage of the standard EDF-statistics 
to test for exponentiality and secondly we did a power study 
for different tests, by the Monte Carlo method. This is a 
mare of a study done by M. A. Stephens, for usage of the 


EDF-statistic for goodness-of-fit in general. 
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CHAPTER I 


PSHEO Introduction. 


An important problem in statistics relates to obtain- 
ing information about the form of the population from which the 
sample is drawn. This problem is known as the goodness-of-fit 
problem. Stated in more exact terms we have the following: 


etek 


; | 1 < i <n} be independent, identically distributed 


(i.i.d.) random variables (r.v.'s) with the distribution func- 
tion F . The goodness-of-fit problem is to devise a test of 


the hypothesis 


Hy eres) ba GA) 


where G(x) is a given distribution function. 
The problem can be divided into two parts: 


(a) G(x) is completely specified, for example, it may be the 
exponential distribution with mean 1 on the non-negative 


reals, or 


(b) G(x) is known to be a member of a specified family, for 
example, it may be an exponential distribution on the non- 


negative reals with an unspecified mean. 


The classical test for the goodness-of-fit problem is 
the chi-square statistic, which has several advantages over the 


more modern EDF-statistics (so called because the statistics are 
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based on the empirical distribution function). For example, 
the chi-square statistic can be used for both continuous and 
discrete cases. It is also known how to use the chi-square 
Statistic in both cases (a) and (b). However, in general the 
EDF-statistics give more powerful tests of Hy than ,the chi- 
square and in at least the case when the hypothesised distribu- 
tion is Normal or Exponential, the EDF-statistic can be used in 
case (b). In section 1.2.0 we will define the most common of 
the EDF-statistics, namely, the Cramér-von Mises, the Anderson- 
Darling, the Watson, the Kolmogorov-Smirnov and the Kuiper 


statistics. 


In section 1.3.0 we will show that the EDF-statistics 
can be expressed by some simple formulas for purposes of calcu- 
lation. In so doing it becomes clear that the tests using these 
statistics are equivalent to testing whether {X, eal lene 


has a distribution G (completely specified) or to testing if 


{F (X,) [el <i Nie has a unitormm (Oj) adistribution. 


If G is not completely specified, we must estimate 


the parameters 6,,°°°,8, (say) by T) = ty (X 709%) fee 
T, = ee a gee oe) (denoted collectively by T) and then make the 
transformation 

Z, = G(X, |Z) Bet ROO are) 9 


We find the Z's to be neither independent nor uniformly 


distributed, and in general the distribution of Zs will depend 
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From now on we will only consider r.v.'s with a density 


function. 
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then the empirical distribution function based on the sample 
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and hence Ff) is the proportinn of the Xi oy ish) es -rn | 


which are less than or equal to x. 


The EDF-statistics are based on the differences between 
FX) and the hypothesised distribution F . For this to make 
sense we need F(x) to converge to F , in some sense, under 


the null-hypothesis. Hence the following theorem: 


eer Dter Le Theorem. (Glivenko-Cantelli Theorem). 
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sup |F (x) - F(x) | 70 3 
—~O< KX <0 


(We omit the proof. For reference see, for example, K.L. Chung 


"A course in probability theory".) 


In 1928 Cramér suggested the following test function 
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where K(x) is a suitable non-decreasing weight function. Von- 
Mises independently made an equivalent suggestion. Smirnov [16 ] 


gave the modification 
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Wheromes {jaro Oncetas Lioisea non-negative wieghtsfunction. In 


case Y¥(t) = 1 , we will call this the Cramér-con Mises statis- 


tic and denote it by we i.e. 


2 “i 2 
Wa =n | (F(x) = F(x))- daF(x) : 
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4 is unknown; however, the limit 


The exact distribution of W 


distribution as n> ° was found by Smirnov. 


Anderson and Darling [1] considered the weight function 
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and call it the Anderson-Darling statistic. Anderson and 
Darling also found the limiting distribution of ay, and 
Marshall [10] showed that the asymptotic critical values are 


valid for n> 5 for all practical purposes. 


It is clear that the Anderson-Darling statistic is more 


sensitiverton.differences-inethe) "tails" between Fs) and 


F(x) than the Cramér-von Mises statistic since Y¥(t) = otic 
LSeminimum for “tc = > and increases as t7>0O or 1. 


Another variation of. the’! Cramér-statistic was “introduced 


by Watson [18], and is defined by 


2 m we 2 
UL, =n | (F (%) oF (x) - | (FL (x) -F (x) ) dF (x) ) eam  p 


=0O —=—0O 


We will call this the Watson statistic. Watson also found the 
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In 1933 Kolmogorov L6] suggested a test of He based 
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They are called the Kolmogorov-Smirnov statistics. 
Smirnov [15] simplified the derivation of the limit distribu- 


tion, and also tabulated De : 


Kuiper [7] suggested the modification 


We will call this the Kuiper statistic. The distribu- 


tion. functions for ayn V,, was also found by Kuiper. 


i 0 ysome: Practical Simplications. 


The definitions in section 1.2.0 can be expressed for 


computation more simply 
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but tedious integration 


is monotone increasing. 


as in lemma 1.3.1. The formulas in 


ue can be obtained by straight-forward 


and the rest follow directly since F(x) 


Therefore, the proof is omitted. 


a Ore nL Lemma. 

Let {X, | 1 < i <n} be i.i.d r.v.'s with the common 
continuous distribution function F(x) . Let {X (3) ]2 rok Se; 
denote the order statistics of {x, | 1 < i<n} with 
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For each of these statistics the region of rejection 


for testing Ho consists of all sufficiently large values of 


the statistic. 


Poe Lemma. 
Let {X, | lsi<n} be i.i.d. r.v.'s with density 


tion f , then {Z, | l<ixn} defined by 


oe 
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Zs = (, £f(t)dt = F(X; ) 


are independent with uniform (0,1) distribution. 
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Hence P(X(;)) , 1 = 1,...,n , behaves as an ordered 
sample from uniform (0,1) distribution under the null-hypothesis, 
and therefore the distribution EDF-statistics in lemma 1.3.1 are 


all independent of the distribution of the X;'S : 


eta Studentized Integral Transformation. 


Lotion Shegiels isniieshe cid sdeenisswith jdensiity func- 
ELON nes ol (%y,0.)) 9; 05 Unknown, $k e= olst.cun, Sabene wuet 05 be estimated 
by T, = t, (X) for i-=1,...,s . Then the studentized integral 


transformation we define by 
x 
Z = | £(u, 2) dues x, Da. 


In general the distribution function of Z will depend 
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If it so happens that the joint distribution function 


of Zs = F(X, ,T) , 1 = 1,...,n does not depend on the parameters 


6 , we define the studentized Cramér-von Mises statistic by 
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Similarly we can define studentized EDF-statistics for the 


Anderson-Darling statistic, the Watson statistic, etc. We will 


denote the studentized EDF-statistics by ay # ue y {heel 
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In this chapter we will consider the goodness-of-fit 
problem, when the hypothesised distribution is exponential 
with the scale parameter unknown, and in one case when the 


location parameter is also unknown. 


There is a fairly extensive literature on testing 
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specified distribution, in this case uniform. Instead of 
testing the original data for exponentiality we will 
transtorm the data and test for uniformity., A test to do 


so has already been discussed in section 1.3.0. 
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The properties we are most interested concern the 
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The proof is trivial, and will therefore be omitted. We 


need the following result to prove the completeness. 


2.2.6 Lemma 
Let {X, | 1SiSn} be71.1,dear.V. 5S with density tunction 


exp(6,0). Define w by 


then w and Kerr) are independent and ou has a chi-square 


distribution with (2n-2) degrees of freedom. 


Proof 


We can write w as 


amare (Leer al2jee ey ps ke (jr 


using again the notation of lemma 2.2.2 we have 


Substitution in the formula’ for..w ‘gives us 


n i) 

w= 2 yy Y;+n6-nY, —no 
j=l i=l 
n J 


fetsead ysteieb Sty a Mis. bak. bod 
mt 

ee 

hin : 


dpeapa® uae = i 


es 
anarperts B eed 5 bas tnehaegsbat ox “ny bas wo dt 


\mbbopxt to ebonpeb (S4AS) ddéw e stb 
am . 


a 

: ra — 

es w otiswaso OW © 
\ aa 


tageh stele eee des tx) 
svedisw S228 annet 20 nottezoa edd 


Cp2)™ 


: ; i 
- wet “alt 
hie, al 
ag esvip w ok sivmrot ods mi ni 


- 29 - 


n 
= 2f ds 
i=2 
where d; is defined in lemma 2.2.3. Since dy is independent 
On do,---,d, it follows that X01) and w are independent. 


Since 2di has a chi-square distribution with 2 degrees 


SL verecdonm, Or vall, i, = has a chi-square distribution 


with (2n-2) degrees of freedom. 


Ze) eeCOLOLl ary. 


£no has a chi-square distribution with (2n-2) degrees 


of freedom. 


2.2.8 Theorem. 
Let {xX, |1Sisn} be dvi dana. bSewich density; function 
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Proof 
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By lemma 2.2.6 ow has a chi-square distribution with 
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Hence by two-dimensional uniqueness theorem for Laplace 


transform 
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Proof. 


To prove § is unbiased, we have 


‘ nE ( (1 )) -E(x) 7 n(= +0) -(o+8) 
n-l n-l 


Hence the lemma follows from theorem 2.2.8. 


Writing 


then the numerator and denominator are independent by 
lemma 2.2.6. Since 
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it follows that (n-1)R has a F distribution, and hence the 


density of R is given by 
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= 0 otherwise. 
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The uniformly most powerful unbiased test of size a 
for the hypothesis that he a given that the sample is 
from an exponential distribution with o unknown, is defined 


DVetier critical regions: t < O0eand r>r. where Q@ = GENS ce 


2.3.0 Transformation from exponential to uniform distribution. 
Let y = (yy r++ rn) be n independent observations from 

some distribution F. Suppose J and L are two transformations 

with the following property: applying J (or L) to y, say 

Ne ye (ZyreeerZey_yy) then Zsacts as ai sample froma 

uniform (0,1) distribution iff.y,is a sample from exp(0,o). 

Hence, it is equivalent to test if the y, (1Sisn) are a 

sample from exp(0,0) or to test if the Z.(1Si<n-1) are 


uniformly distributed between (0,1). 


This is particularly useful when o is unknown, since 
the distribution of Z.(1Si<n-1) is completely known and we 


can use lemma 1.3.1. 
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we will now prove that transformations J and L have the 
desired property, i.e. that Z,'s are independent and Z is 
uniformly distributed iff y is a random sample from exp(0,°) 


under some restrictions. 


Transformation J 

The transformation J has been considered by Seshadri, 
Csorgo and Stephens and most of the proof is taken from their 
paper. We first need to prove some lemmas before the 


required result in Theorem 2.3.5. 
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: 4 (¥%y) = olyyt..-ty,) 


We have 
i=l 
whose solution is ¢$¢(x) = e** where a is an arbitrary constant. 
Hence 
ay, 
ply;) = plo)e ia asre 


Since p(y) is a density function with mean equal to l, 


This completes the proof of the lemma. 
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This is the density of the spacings of n-l observations 
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density function p,;p(y)>0 for all yel0,~), and E(Y;) = 0o>0. 
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then a5 are independent and uniformly (0,1) distributed iff 


Y, are exp(0,0) distributed. 


Proof of Sufficiency 
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and is easy to check that 4-2 have the same distribution as 
the Ge1jce ordered statistic from a sample of size n from 
unitorm (0,1) distribution, and <sovon. = Hence the result 


follows from theorem 2.3.5. 


One disadvantage with transformation J(L) is that 
25 (2) depends on the order of Teen i.e. the test is not 
invariant under permutation of the sample. However, this 
problem can be solved in the following way. Recalling theorem 
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Y,'s are independent and exp(0,0) distributed. This trans- 
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The least squares method requires that we minimize 


the scalar sum of squares 


for variation in the components of 6. 
The generalized case is the same except that 
Var(e) = o°V 
where V is an (n x n) completely specified matrix. 


Lloyd [9] considers the special case of estimating 


the location and scale parameters from the order statistics. 
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Then the generalized least squares estimate is given by 
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raw Definition. 


The W-statistic is defined by 


where W is to be used as a two-tailed statistic. 


Note that W is a ratio of two estimates of variance, 


except for a factor of N . W can also be written as 
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Properties of W-statistic 
2.4.2 Lemma. 
W is scale and location invariant. The proof follows 


immediately from the definition. 


2.4.3 Lemma. 
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Hence the minimum is equal to Cael and the lemma is proven. 


In the next lemma we will use the following result 
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Let the family of possible distributions of X 
be D = {P., Ve 2}. Let T be a sufficient statistic for 
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boundedly complete. Then a statistic V is independent 
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2.4.4 Lemma. 
Let {X,|l<i<n} be i.i.d. r.v.'s from exp(6,0) then the 


w-statistic is independent of Xeqy and xX. 


Proof 


Follows immediately from Basu result. 
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R and W are independent where R is defined in 2.2.10. 
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One disadvantage of using the EDF-statistic has been 
the need for extensive tables. Stephens [13] has suggested 
that for Gach test statistic, T say,“a simple modification, 
T*, is given and T* is compared with the given percentage 
points. We will discuss this modification in section 3.2.0. 
In section 3.3.0 we will consider another approximation to 
the percentage points, such that standard chi-square tables 


can be used. 


In section 3.4.0 we will discuss the power of the 
EDF-statistics and compare these with Shapiro and Wilks' 


W-statistic. 
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of the EDF-statistics. Stephens has suggested a method 
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after using the transformation J or L, the following 


modification was suggested: 


StacisticeL 


A 


Modified T* 


pt” = p*(Yn+0.12+0.11/¥n) 

D™” = D-(/n#0.1240.11//n) 

D’ = D, (vn+0.12+0.11/Yn) 

V* = V, (/n+0.155+0.24/yn) 

We = (W2-0.4/n+0.6/n~) (1.0+1.0/n) 
Ut = (U*-0.1/n+0.1/n*) (1.0+0.8/n) 
Ax) =e LOwgalien5 


The percentage points for T* are given in Table I. 


The percentage points for the EDF-statistics, when the 


samples were taken from an exponential distribution with 


unknown mean, have been 
Dovterors (96.1. did sens 
statistic, and Stephens 


other EDF-statistics. 


For those values of 


found, using the Monte Carlo method. 
first for the Kolmogorov-Smirnov 


[13] completed the tables for the 
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EDF~statistics are known, the simulated values agree very 
closely with both Lillefors' and Stephens' results. 

Similarly, for the case when F(x) is completely specified, 
Stephens [13] suggested the approximation Te, to simplify 


the tables. In this case the modifications are given as 


follows: 
Statistic T Modification T* 
D., D* = (D_-0.2/n) (Y¥n+0.26+0.5//n) 
va Whe (V-0.2/n) (Yn+0.24+0.35//n) 
if) Ry in er 
We Ww* = W (1+0.16/n) 
oe) Pea pe. 
D U* = U_, (4+0.16/n) 
~ a2 Lae 
i A A, (1+0.6/n) 


~ 


The percentage points for T* are given in Table II. 


2.3.0 Chi-square approximation. 


AY sufficiently good japproxamation of T* can be found 
in the form 


~ 


T* (a) = atbx< (ct) 


where the constants a, b and V are obtained by matching 


the moments of T*. The moments of T* were obtained by the 
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Monte-Carlo method. Using this method, a standard chi- 
square table can be used forall EDF-statistics. In 
Table III we give the first four moments of T*. The 


constants a, b and v are given by: 


D* (a) 


2 
0.01740 .0343X.,, (a) 


V*(a) = -0..336+0.0295x2 


50 '%) 


a 2 
W* (a) = 0.0460+0.0466X, (a) 


~ 2 
Us (a) 0.026540 .0266x, (a) 


where x2 (a) is the upper tail percentage point at level a, 


GOfea chi-square distribution with Vv df. 


Although this appears to be an approximation of an 
approximation, the results are very accurate. Fora 


comparison, see Stephens [13]. 


3.3.0 Power of the EDF-statistic. 

In Chapters I and II we have discussed five different 
ways of using EDF-statistics for testing for exponentiality, 
i.e. 

a) Integral transformation 
21) Veoetransformation 
ZL LD)ML Ltranstormation 
1V) 0 0’ stranstormation 


v) L' transformation 


eR 


(0) gax@es0.0%9€6.0- = (u)*¥ 


(b) x2a80.040980.0 = (new ae 
(1) $xaa80,042850.0 = (28 | a - 


.o level 36 staked oviiabszeq ftps soqau sits ak qs ; sed " 


2 6 ve dati notdudixtert susupe-ido 5 


rs 
ne Yo notdsotionaue ms ado? ezasage aii Mpwodain <a : 
5 107 -BFE=U995 a th ous adtuast ans abs ia 7 
Rod Paacahee ve onto 


We have also considered the 
i) W-statistic 


12) T, ~statistic 


We now have the following problems: 
1) How does the power compare between the different 
EDF-statistics? 


2) What transformation gives the greatest power? 


To find the power, we used the Monte-Carlo method 
and selected 5,000 random samples from the following 
distributions: 

a) Half normal (i..e., absolute value 
of samples from N(0,1) 

b) Half Cauchy 

Ce Una forme (0, an) 

dad) F(x) = (Guess AP 

e) F(x) = Tce) 


f) Chi-square (1-df) 


and then tested for exponentiality. Table IV gives the 
results of this study. The study indicates that the 
Shapiro-Wilks W-statistic has greater power than any of 

the EDF-statistics, with the exception of the case where 

the sample comes from a chi-square distribution. As expected 
we has greater power than oe , Since Wie LS; a “Sum of 


discrepancies. More complete tables will be published by 


Stephens. 
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TABLE 11 


Modified EDF-Statistic, When F(x) Completely Known 


Tx 
15 

Dia Dae .973 

D* 1,138 

vs 1.537 

we 0.284 

Ux 0.131 

Ax 1.61 


TSource Stephens [14] 


Percentage Points for T* 
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TABLE 27 


Modified EDF-Statistic for Test of Exponentiality 


Tx 


D* 


Vx 


wWw* 


Ux 


A* 


T source Stephens [14] 


ILS 


0.926 


1.445 


0.149 


OnEt2 


02922 


Scale Parameter Unknown 


Percentage Points for T* 
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TABLE 37 


Estimated Moments of Null Distributions: 


Monte Carlo Results, Not Smoothed 


Statistic Moments 


n my m, m5 m 
6 0.673 0.485 On37 5 0.308 
10 687 1507 400 337 
YvnaD ~=-20 703 esi 431 375 
50 Hie 2545 448 395 
100 570 ey 462 sik, 
6 0.094 0.013 0.0024 0.0006 
10 094 013 0026 0007 
fe oR 095 014 .0029 . 0009 
50 094 013 0027 . 0008 
100 094 013 0027 . 0007 
6 1.084 i282 1.466 1.824 
10 1.106 i285 1.563 1.989 
vnV—-.20 Pao 19355 1.696 Ras 
50 1.154 1.400 760 |S 
100 1.169 1.433 1.843 2.483 
6 0.074 0.007 0.001 0.0002 
10 .074 .007 001 . 0002 
u" 20 .074 .007 001 . 0002 
50 NOWE .007 001 . 0002 
100 2013 .007 001 . 0002 


+ Source Stephens [13] 
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TABLE 4A 


Power Comparisons; Test for Exponentiality, ® Unknown. The Table Gives 
the Percentage of Samples Significant, When the Population is as Shown, 
and the Sample Size is 20. The Test is at the 10% Level. The EDF- 
Statistics are Calculated Using Studentized Integral Transformation. 
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TABLE 4B 


Power Comparison; Test for Exponentiality, 6 Unknown. The Table 
Gives the Percentage of Samples Significant When the Population is 
as Shown, and the Sample Size 20. The Test is at 10% Level. The 
EDF-Statistics are Calculated Using Transformation L. 
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TABLE 4C 


Power Comparison. Test for Exponentiality, ® Unknown. The Table 
Gives the Percentage of Samples Significant, When the Population is 
as Shown, and the Sample Size 20. The Test is at 10% Level. The 
EDF-Statistics are Calculated Using Transformation L'. 
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